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Abstract 

Motivated by the relationship between the eigenvalue spectrum of a network and the behavior of 
dynamical processes evolving in it, we propose a distributed iterative algorithm in which a group of n 
autonomous agents self-organize the structure of their communication network in order to control the 
network's eigenvalue spectrum. In our algorithm, we assume that each agent has only access to a local 
('myopic') view of the network around it and that there is no centralized coordinator. In each iteration, 
agents of the network perform a decentraUzed decision process in which agents share hmited information 
about their myopic vision of the network to find the most beneficial edge addition/deletion from a spectral 
point of view. We base our approach on a novel distance fimction defined in the space of eigenvalue 
spectra that is written in terms of the spectral moments of the Laplacian matrix. In each iteration, agents 
in the network run a greedy algorithm to find the edge addition/deletion that minimized the spectral 
distance to the desired spectrum. The spectral distance presents interesting theoretical properties that 
allow an elegant and efficient distributed implementation of the greedy algorithm using distributed 
consensus. Our distributed algorithm is stable by construction, i.e., locally optimizes the network's 
eigenvalue spectrum, and is shown to perform very well in practice. 
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I. Introduction 

A wide variety of complex networks composed of autonomous agents are able to display a 
remarkable level of self-organization despite the absence of a centralized coordinator [[H, 
For example, the intricate structure of many biological, social and economic networks, emerges 
as the result of local interactions between agents aiming to optimize their local utilities jSl. In 
most real cases, these agents have only access to myopic information about the structure of the 
network around them. Despite the limited information accessible to each agent, most of these 
"self-engineered" networks are able to efficiently satisfy their functional requirements. 

The behavior of many networked dynamical processes, such as information spreading, synchro- 
nization, or decentralized coordination, is directly related to the network eigenvalue spectra [4J. 
In particular, the spectrum of the Laplacian matrix of a network plays a key role in the analysis 
of synchronization in networks of nonlinear oscillators Q, [|6l, as well as in the behavior of 
many distributed algorithms and decentralized control problems dH, flU. Motivated by the 
relationship between a network's eigenvalue spectrum and the behavior of dynamical processes 
evolving in it, we propose a distributed iterative algorithm in which a group of autonomous agents 
self-organize the structure of their communication network in order to control the network's 
eigenvalue spectrum. The evolution of the graph is ruled by a decentralized decision process 
in which agents share limited information about their myopic vision of the network to decide 
which network adjustment is most beneficial globally. 

Optimization of network eigenvalues has been studied by several authors in both centralized 
[[Toll , [fTTTl . [fT2l and decentralized settings [fT3l . In these papers, the objective is usually to 
find the weights associated to the edges of a given network in order to optimize eigenvalues 
of particular relevance, such as the Laplacian spectral gap or spectral radius (i.e., the second 
smallest and largest eigenvalues of the Laplacian matrix, respectively). In contrast to existing 
techniques, we propose a distributed framework where we control the so-called spectral moments 
of the Laplacian matrix by iteratively modifying the structure of the network. We show that the 
benefits of controlling the spectral moments, instead of individual eigenvalues, lies in lower 
computational cost and elegant distributed implementation. The performance of our algorithm is 
illustrated in nontrivial computer simulations. 

The rest of this paper is organized as follows. In Section [11} we review terminology and 
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formulate the problem under consideration. In Section Ini} we introduce a decentralized algorithm 
to compute the spectral moments of the Laplacian matrix from myopic views of the network's 
structure. We also introduce a novel perturbation technique to efficiently compute the effect of 
adding or removing edges on the spectral moments. Based on these results, in Section lIVl we 



propose a distributed algorithm in which a group of autonomous agents modify their network of 
interconnections to control of the spectral moments of a network towards desired values. Finally, 
in Section IVl we illustrate our approach with several computer simulations. 



II. Preliminaries & Problem Definition 

A. Eigenvalues of Graphs and their Spectral Moments 

Let Q = (V, £) be an undirected graph, where V = {1, . . . , n} denotes a set of n nodes and 
£ CV X V denotes a set of e undirected edges. If G £, we call nodes i and j adjacent 
(or first-neighbors), which we denote by z ~ j. We define the set of first-neighbors of a node i 
as Mi = {j E V : E £}. The degree di of a vertex i is the number of nodes adjacent to 
it, i.e., di = lA/il^An undirected graph is called simple if its edges are unweighted and it has 
no self-loop^ A graph is weighted if there is a real number associated with every edge. More 
formally, a weighted graph "H can be defined as the triad "H = (V,£^, W), where V and £ are 
the sets of nodes and edges in "H, and W = {wij E M, for all E £} is the set of (possibly 
negative) weights. 

Graphs can be algebraically represented via matrices. The adjacency matrix of a simple graph 
Q, denoted by Ag = [aij], is an n x n symmetric matrix defined entry-wise as aij = 1 if nodes i 
and j are adjacent, and aij = otherwise. Given a weighted, undirected graph "H, the weighted 
adjacency matrix is defined by W-^ = [wij], where Wij is the weight associated to edge E £ 
and Wij = if i is not adjacent to j. We define the degree matrix of a simple graph Q as the 
diagonal matrix Dg = diag (di). We define the Laplacian matrix Lg (also known as combinatorial 
Laplacian, or Kirchhoff matrix) of a simple graph as Lg = Dg — Ag. For simple graphs, Lg is a 
symmetric, positive semidefinite matrix, which we denote by Lg ^ [TBI . Thus, Lg has a full set 
of n real and orthogonal eigenvectors with real nonnegative eigenvalues = Ai < A2 < ... < A„. 

'We define by |X| the cardinality of the set X. 
self-loop is an edge of the type 
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Furthermore, the trivial eigenvalue Ai = of Lg always admits a corresponding eigenvector 
vi = (l,!,...,!)"^. The algebraic multiplicity of the trivial eigenvalue is equal to the number of 
connected components in Q. The smallest and largest nontrivial eigenvalues of Lg, A2 and A„, 
are called the spectral gap and spectral radius of Lg, respectively. 

Given an undirected (possibly weighted) graph Q, we denote its Laplacian spectrum hy S (Q) = 
{Ai, A„}, and define the k-th Laplacian spectral moment of ^ as, [fT4|: 

1=1 

The following theorem states that an eigenvalue spectrum is uniquely characterized by a finite 
sequence of moments: 

Theorem 2.1: Consider two undirected (possibly weighted) graphs Gi and G2 with Laplacian 
eigenvalue spectra S {d) = {X^^^ < ... < X^^^ and ^2 (Gi) = {A^^ < ... < A^f^}. Then, 
Af ) = Af ^ for all 1 < i < n if and only if (d) = (G2) for < A; < n - 1. 

Proof: In the Appendix. ■ 

In the rest of this paper we will focus on the spectrum of the graph Laplacian matrix Lg and 



its spectral moments, which we denote by (Lg). In this case. Theorem 2.1 , implies that the 
Laplacian spectral moment of a graph on n nodes is uniquely characterize by the sequence of 
n — 1 spectral moments {nik iLg))^Z[. It is worth remarking that two nonisomorphicj^ graphs 
Qi and Q2 can present the same eigenvalue spectrum [|T6l , in which case we say that Qi and Q2 
are isospectral. In other words, the eigenvalue spectrum of a graph is not enough to characterize 
its structure. On the other hand, as we shall show in Section |III} there are many interesting 
connections between the structural features of a graph Q and the spectral moments of its Laplacian 
matrix, (Lg). 

B. Local Structural Properties of Graphs 

In this section we define a collection of structural properties that are important in our deriva- 
tions. A walk of length k from node ii to node z^+i is an ordered sequence of nodes (ii, 12, ik+i) 
such that ij ~ ij+i for j = 1,2, One says that the walk touches each of the nodes that 

'Two simple graphs Qi and Q2 with adjacency matrices Ag-^ and Ag^ are isomorphic if there exists a permutation matrix Pn 
such that Ag^ = PnAg^P^. 
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comprises it. If ii = ik+i, then the walk is closed. A closed walk with no repeated nodes (with 
the exception of the first and last nodes) is called a cycle. Given a walk p = (zi, Z2, ik+i) in 
a weighted graph T-L with weighted adjacency matrix Wn = [wij], we define the weight of the 
walk as, u (p) = Wi^i^Wi^i^...Wi^i^^^. 

We now define the concept of local neighborhood around a node. Let S{i,j) denote the 
distance between two nodes i and j (i.e., the minimum length of a walk from i to j). By 
convention, we assume that 5 (i, i) = 0. We define the r-th order neighborhood Qi^r = {■^i,r,£i,r) 
around a node i as the subgraph Qi-r C Q with node-set A/^^r = {j eV : 6 < r}, and edge- 
set £i,,. = {{v,w) E £ s.t. v,w E Mi^r}- Given a set of k nodes /C C V, we define Qic as the 
subgraph of Q with node-set V (Gjc) = ^ and edge-set £ (Qjc) = E £ s.t. i,j E /C}. We 

define Lg (/C) as the k x k submatrix of Lg formed by selecting the rows and columns of Lg 
indexed by /C. In particular, we define the Laplacian submatrix Lj ,. = Lg (Mi^r)- 

We say that a structural measurement is local with a certain radius r if it can be computed 
from the set of local neighborhoods {Qi^r, i = l,--.,n}. For example, the degree sequence of 
^ is a local structural measurement (with radius 1), since we can compute the degree of each 
node i from the neighborhood A/^ i. In contrast, the eigenvalue spectrum of the Laplacian matrix 
is not a local property, since we cannot compute the eigenvalues unless we know the complete 
graph structure. One of the main contributions of this paper is to propose a novel methodology 
to extract global information regarding the Laplacian eigenvalue spectrum from the set of local 
neighborhoods. 



C. Spectral Metrics and Problem Definition 

As discussed in Section |l} our goal is to propose a distributed algorithm to control the eigen- 
value spectrum of a multi-agent network, via its spectral moments, by iteratively adding/removing 



edges in the network; see Section II-B For this, we define the following spectral distance between 
two graphs Ga and Gb, with spectra Sa = {A-"'*}"^i and Sb = {Xf^}'^^^, a^ 



71-1 



dM {Sa, Sb) = ^ (mk (Go) 

k=l 



rrik (Gb 



(2) 



''Note that dM is a distance in the space of eigenvalue spectra, but not in the space of graphs, since we can find nonisomorphic 
graphs that are isospectral. 
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According to Theorem 2.1 two graphs are isospectral if their first n — 1 spectral moments 
coincide; thus, rf^/ in ([2]) is in fact a distance function in the space of graph spectra. We further 
define the spectral pseudometri^ 

dK {Sa. S,) = J2 (^fe {GaY^' - (Gfe)'/') , (3) 

fe=l 

iox K < n — 1. The benefit of using the spectral pseudodistance versus other spectral distances 
is due to the fact that, as we shall show in Section [Hi} we can efficiently compute the first 
K spectral moments of the Laplacian matrix from the set of local Laplacian submatrices with 
radius yK/2\, i.e., {i^i,[ft:/2j , ^ G V}. In other words, assuming that each agent has access to 
the Laplacian submatrix associated to its neighborhood with radius r, we shall show how to 
distributedly compute the first 2r + 1 Laplacian moments of the complete graph Lg. With the 
notation defined above, we can rigorously state the problem addressed in this paper as follows: 

Problem 1: Given a desired spectrum S* = {\*}^^^, find a simple graph Q* such that its 
Laplacian eigenvalue spectrum, denoted by S (G*), minimizes dx (S (G*) , S*). 

Finding a simple graph with a given (feasible |^ eigenvalue spectrum is, in general, a hard 
combinatorial problem, even in a centralized setting. In this paper, we propose a distributed 
approximation algorithm to find a graph with a spectrum 'close to' S* in the dx pseudometric. 
In our algorithm, a group of agents located at the nodes of a network iteratively add/remove 
edges to drive the network's eigenvalue spectrum towards the desired spectrum. In each iteration, 
the set of agents perform a decentralized decision process to find the most beneficial edge 
addition/deletion from the point of view of the global eigenvalue spectrum. 

To formulate our algorithm, we first need to define the edit distance ds (Ga, Gb) between 
two graphs Ga and Gb, which is the minimum number of edge additions plus edge deletions 
to transform Ga into a graph that is isomorphic to Gb- To approximately solve Problem [T] in a 
distributed way, we propose the following iteration to determine a sequence of graphs {G{t)}t>o, 

pseudometric is a generalization of distance in which two distinct points (in our case, two distinct spectra) can have zero 
distance. 

*We say that an eigenvalue spectrum is feasible if there is a simple graph whose Laplacian matrix presents that spectrum. 
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Starting from any graph Qq: 

g{t + l)^ argming dK{Sig),S*) 

s.t. dE{g{t),g) = i, (4) 

A2 (Q) > 0. 

The resuking sequence of spectra {S {g{t))}t>o converges to S* as t grows. The constraint 
dE {G (t) , G {t + ^)) = 1 enforces only single edge additions or deletions at each iteration, 
while the requirement A2 {G (t)) > enforces graph connectivity at all times, which will be 
necessary for the distributed implementation in Section IV Note that the Iteration (|4]) typically 



requires global knowledge of the network structure. In this paper, we propose a computationally 
efficient, distributed algorithm in which agents in the network solve Q using only their local, 
myopic views of the network structure. In particular, we shall show how the set of agents can 
compute, in a distributed fashion, the effect of an edge addition/deletion on the first 2r + 1 
Laplacian moments. Furthermore, we shall also propose a distributed algorithm to find the edge 
addition/deletion that minimizes the resulting value of the spectral pseudodistance to S* . Before 



we describe the implementation details in Section IV we first provide the theoretical foundation 



for our approach in Section III 



Remark 2.1 (Convergence): Several remarks are in order. First, note that it is not always 
possible to find a simple graph that exactly match a given eigenvalue spectrum. Second, the 
spectral pseudometric (S (G) , S*) may present multiple minima for a given 5**. These minima 
could correspond, for example, to several isospectral graphs matching the desired spectrum S* 
|fT6ll . Therefore, iteration (|4]) may converge to different isospectral graphs depending on the initial 
condition Go- Third, iteration (|4]) finds the most beneficial edge addition/deletion in each time 
step, hence, this greedy approach may get trapped in a local minimum. In practice, we observe 
that in our numerical simulations the spectra of these local minima are remarkably close to those 
of the desired spectrum. 

III. Moment-Based Analysis of the Laplacian Matrix 

In this section, we use tools from algebraic graph theory to compute the spectral moments 
of the Laplacian matrix of G when only the set of local Laplacian submatrices {Li ^, i G V} is 
available. As a result of our analysis, we propose a decentralized algorithm to compute a truncated 
sequence of Laplacian spectral moments via a single distributed averaging. Furthermore, we also 
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present an efficient approach to compute the effect of adding or deleting an edge in the Laplacian 
spectral moments of the graph. Particularly useful in our derivations will be the following result 
from algebraic graph theory [fT4l: 

Lemma 3.1: Let "H = iV,£, W) be a weighted graph with weighted adjacency matrix Wy^ = 
[wij]. Then 

where Pi^k {'H) is the set of closed walks of length k starting and finishing at node i in the 
weighted graph H. 

A. Algebraic Analysis of Structured Matrices 

Consider the symmetric Laplacian matrix Lg of a simple graph Q = {y,S). We denote by 
Qi,r = iJ^i,r,£i,r) the neighborhood of radius r around node i and define the local Laplacian 
submatrix Li^^, as the submatrix Lg (Mi^r), formed by selecting the rows and columns of Lg 
indexed by the set of nodes A/i , . By convention, we associate the first row and column of the 
submatrix Lj with node i E V, which can be done via a simple permutation of rows and 
columns]^ For a simple graph Q with Laplacian matrix Lg, we define C (Q) as the weighted 
graph whose adjacency matrix is equal to Lg. In other words, C (Q) has edges with weight — 1 
for E S (Q), for ^ S (Q), and di for all self-loops i E V (Q). We also define 
Tii^r as the weighted subgraph of C{Q) with node set Mi^r, containing all the edges of C{Q) 
connecting pairs of nodes in A/^ r (including self-loops). Notice that, according to this definition, 
the weighted adjacency matrix of "Hj ,, is equal to Lj ^- 

In this paper, we assume that each agent in the network knows the structure of its local 
neighborhood Qi^^, for a fixed r. Therefore, agent i has access to the local Laplacian submatrix 
Lj r- The following results allows us aggregate information from the set of local Laplacian 
submatrices, {Ljrjjgy, to compute a sequence of spectral moments of the (global) Laplacian 
matrix Lg. 

Theorem 3.2: Consider a simple graph Q with Laplacian matrix Lg. Then, for a given radius 

^Notice that permuting the rows and columns of the Laplacian matrix does not change the topology of the underlying graph. 
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r, the Laplacian spectral moments can be written as 



1 " 



n 

i=l 

foxk<K = 2r + l. 

Proof: Since the trace of a matrix is the sum of its eigenvalues, we can expand the fc-th 
spectral moment of the Laplacian matrix as follows: 

mk {Lg) = -Trace (Lg) 

n 

1=1 

Therefore, since Lg is the weighted adjacency matrix of the Laplacian graph C{Q), we have 



(from Lemma 3.1 1 

1 

ruk {Lg) = - ^ ^ w (p) , (6) 

where the weights u (p) are summed over the set of closed walks of length k starting at node i 
in the weighted graph C{Q). 

For a fixed value of k, closed walks of length k in C{Q) starting at node i can only touch 
nodes within a certain distance r (k) of i, where r (k) is a function of k (see Fig.[T]). In particular, 
for k even (resp. odd), a closed walk of length k starting at node i can only touch nodes at 
most k/2 (resp. [k/2\) hops away from i. Therefore, closed walks of length k starting at i are 
always contained within the neighborhood of radius [k/2\. In other words, the neighborhood 
Qi^r of radius r contains all closed walks of length up to 2r + 1 starting at node i. Therefore, 
for A; < 2r + 1, we have that 

where "Hj,. is the weighted graph whose adjacency matrix is equal to the local Laplacian 
submatrix Lj ^ (notice that, by convention, we associate the first row and column of with 
node i). Therefore, according to Lemma [TT] we have 

E ^iP)=[Ll],,,- (V) 

pePi,fc(W(L,,,)) 

Then, substituting (|7]) into ([6]), we obtain the statement of our Theorem. ■ 
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Fig. 1. Cycles Ce and C7, of lengths 6 and 7, in a neighborhood of radius 3 around node i. 



Remark 3.1 (Distributed computation of spectral moments): Since every node i has access to 
its local neighborhood it is possible to compute the first 2r + 1 moments via a simple 
distributed averaging of the quantities { [-^^V] ii}«ev> Q- This averaging efficiently aggregates 
local pieces of local structural information (described by the local Laplacian submatrices) to 
produce a truncated sequence of spectral moments of the (global) Laplacian matrix. This is an 
useful result for the analysis of complex networks for which retrieving the complete structure 
of the network can be very challenging (in many cases, not even possible). 



Based on Theorem 3.2 we propose a distributed algorithm to compute a sequence of 2r + 1 
spectral moments of Lg from local submatrices Lj as described in Algorithm [T] Note that, 
computing the spectral moments via ([5]) is much more efficient than computing these moments 
via an explicit eigenvalue decomposition for many real-world networks. In most real applications, 
the Laplacian matrix representing the network structure is a sparse graph for which the number 
of nodes in the neighborhood Mi^r is very small compared to n, for moderate values of r. 

B. Moment-Based Perturbation Analysis 

In this section, we use spectral graph theory to compute the effect of adding or deleting 
an edge on the spectral moments of the Laplacian matrix. Traditionally, the effect of a matrix 
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Algorithm 1 Decentralized moment computation 

Require: Local Laplacian submatrices Li,,, for all nodes i E V; 

1: Each node i G V computes a vector fi^ = (/i^ i, /i^ 2, f^i,3, /^i,2r+i)^' where fx^ ,^ = [Lf ,,] 

2: Using distributed averaging, compute the following vector of averages: 

1 

m2^+i (Lg) = - V Mi 
n ^-^ 

i=l 

1 " T 
i=l 

= (mi (Lg) , 1712 (Lg) , m2r+l {Lg)Y ■ 



perturbation on the eigenvalue spectrum is analyzed using eigenvalue perturbation techniques 
ifTTll . In particular, the effect of adding a 'small' perturbation matrix 5W to an x symmetric 
matrix W with eigenvalue spectrum {cTfcjlLi be approximated, in the first-order, by [fTTll 

5fc -CTk^ul 6W Uk, 

where Uk is the eigenvector of W associated with the eigenvalue ak, and {o^k}k=i the 
eigenvalue spectrum of the perturbed matrix W + 5W. In the case of the Laplacian matrix, 
the perturbation matrix 6W corresponding to the addition of an edge can be written as, 
6W = (cj — Cj) (cj — ej)^, where is the unit vector in the direction of the i-th coordinate. We 
denote by Q + the graph resulting of adding edge to Q, and {\k}k=i is the Laplacian 
spectrum of Q + Therefore, adding edge perturbs the eigenvalues of the Laplacian 

matrix as follows: 

Xk- Xk ~ {ci - Cj) {ci - Cj)^ Vk 

where Vk is the eigenvector of Lg associated to A^, and Vkj is its j-th component. Hence, the 
resulting spectral radius can be approximated as 

Ai ~ Ai + (f i,i - vijf . 

Therefore, computing the effect of an edge addition on the spectral radius using traditional 
perturbation techniques requires computation of the dominant eigenvalue and eigenvector of Lg, 

September 10, 2012 DRAFT 



12 



which is computationally expensive for very large graphs. As an alternative to the traditional 
analysis, we propose a novel approach, based on algebraic graph theory, to compute the effect 
of structural perturbation on the spectral moments of the Laplacian matrix Lg without explicitly 
computing the eigenvalues or eigenvectors of Lg. Furthermore, our approach can be efficiently 
implemented in a fully decentralized manner. 

In our derivations, we use the following result from algebraic graph theory: 
Lemma 3.3: Let H = (V,£^, W) be a weighted graph with weighted adjacency matrix W-^i = 
[wij\. Then 



mk{Wn) = - V uj{p), (8) 
n ^ — ^ 

where Pk (T-L) is the set of closed walks of length k in the weighted graph T-L. 



Proof: This lemma is a consequence of Lemma 3.1 Specifically, we have that 

mk{W(n)) = ^Trace(iyCH)''^ 



n 



where (T-L) = Ujgv-Pi.fe i'H) is the set of all closed walks of length k in 7i (for any starting 
node 2 G V). ■ 



C. Perturbation on the Spectral Moments 

Consider a simple graph Q with Laplacian matrix Lg. We denote by ^ + (z, j) (resp. Q — (i, j)) 
the graph resulting from adding (resp. removing) edge to (resp. from) Q. Consider the sets 
of nodes A/i,, and A/^ being within a radius r from node i and node j, respectively. Let us 
define the following submatrices indexed by the set of nodes in Afi^r U Mjy. 

Ur,{i,j) = Lg (Mi^r U Mj^r) , 

- (M,r U A/'j, r-) , 
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The following lemma allows us to efficiently compute the increment (resp. decrement) in the 
Laplacian spectral moments of Q due to the addition (resp. removal) of edge («, j): 

Theorem 3.4: Given a simple graph Q with Laplacian matrix Lg, the increment (decrement) 
in the A;-th Laplacian spectral moment of a graph Q due to the addition or deletion of an edge 
can be written as 



rrik {Lg±(ij)) - ruk {Lg) = ^ |^Trace (u^^. .^^ - Trace (f/r,(i,i))^^ 



(9) 



for k<2r + l. 

Proof: Consider the weighted Laplacian graphs of Lg, j), and .,), which we 

denote hy'H = C (Q), V,^ = C{Q + and V.^ = C{Q — respectively. (By definition, 

the adjacency matrices of the Laplacian graphs are the Laplacian matrices of the graphs.) Then, 



according to Lemma 3.3 we have that the A;-th spectral moments mj. (Lg), rrik j)) and 

ruk can be written as weighted sums over the sets of all closed walks of length k in 

v., V.^, and "H", as follows, 

mk{Lg) = - V uj{p), 
n ^-^ 



n 

pePfc(W±) 



We define Pji'f {H), P^f {H+), and P^^f {H-) as the sets of closed walks of length k in, 
respectively, "H, "H^, and "H^ visiting only nodes in the set Ni,r LiMj^r- Then, we can split the 
summation in ([8]) for the Laplacian matrices, as follows: 

rUkiLg) = - V u{p) + - V uj{p), (10) 
n ^-^ n ^-^ 

mk{Lg±^i^,)) = ^ "^^P^^n S ^^^^ 

pGPt]fiH±) pePk\piy\n±) 

Notice that, as we illustrated in Fig. [TJ none of the closed walk of length k <2r + l touching 

node i (resp. node j) can leave the neighborhood A/^^r (resp. Mj-r). Therefore, all closed walks 

of length k < 2r + 1 touching either node i or j (or both) are containecj^ in Afi^r U J\fj,r- As 

a consequence, none of the closed walks in P}\pI:'^^ {H) or Pfc\P^^*'"''' ("H^) touches node i or 

*We say that a walk is contained in a set of nodes if it only touches nodes in A''. 
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j. Since addition/removal of edge does not influence those walks not touching i or j, we 
have that 



pePk\Pi]f{n) 



Thus, from ( 10) and (11) we have 



(12) 



Since P^*^"'^ (H) is the set of all closed walks of length k in V, visiting nodes in the set 
Mi^r ^J^j,r, we can apply Lemma [33] to obtain 

- V C0{p) = nik (Lg {M,r U Afj,r)) 



-Trace iUr(i,j)Y 



Similarly, for P^f (u^^ we obtain 

\ E ^(P) = ^Trace(f/4_^.))' 



(13) 



(14) 



Finally, substituting (13) and (14) in (12) provides us with the statement of our theorem. 



Remark 3.2 (Computational cost): According to Lemma 3.4 we can compute the increment 



or decrement in the Laplacian spectral moments (up to order 2r + l) by computing Trace(f/r ( ^'^ 



and Trace(f/^^. ^.j)'^. Notice that the sizes of Ur,(ij) and are {Afi^r UAfj^rl, which is usually 

small for large sparse graphs (and moderate r). 



IV. Decentralized Control of Spectral Moments 



In this section, we integrate the results developed in Section III with a novel technique for 
distributed connectivity verification of edge additions or deletions in order to obtain a distributed 
solution to Problem [T] in the form of (Q, as discussed in Section |II-C[ This relies on the 
assumption that an agent at node i is able to communicate at time slot t with all the agents in 
its first-order neighborhood A/^^i (t) onlyj^ Moreover, we also assume that every agent has only a 
myopic view of the network structure. This means that at time slot t agent i eV only knows the 



'Notice that, since Q (t) is time-dependent, so are the neighborhoods C/i,r (t) = (A/i,r (t) ,£i,r {t)). 
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topology of the neighborhood Qi^r (t), within a particular radius r. This limits the set of possible 
actions that every agent i can take in every step of the iteration (|4]), to be local edge additions 
of non-edges ^ E it) in Qi,,. (t) or local edge deletions of edges ^ S (t) in Qi^i (t). 

In what follows, it will be useful to predetermine the master node for each edge (i, j) G 8 (t), 
which can be arbitrarily chosen from the set of nodes The notion of master node is 

useful to coordinate actions in our decentralized algorithm. The agent located at the master 
node of is the only one with the authority to decide if edge (i, j) is deleted. We denote 
by Viit) the set of edges having node i as its master. In our simulations, we choose this 
set to he Viit) = {(i, j) G | i > Similarly, it is useful to predefine a master node 
for each nonedgep] {i,k) ^ £{t). The agent located at the master node of the nonedge is 
the only one with the authority to decide if edge (i, k) is added to the network. We denote 
by Ai (t) the set of nonedges having node i as its master. In our case, we define this set as 
Ai (t) = {{i, k) ^ 8{t) I k G Mi^rit) and i > k}, where we limit node k to be in Mi^rii), since 
we are only considering local edge additions. 

A. Connectivity-Preserving Edge Deletions 

In a centralized framework, network connectivity can be inferred from the number of trivial 
eigenvalues of the Laplacian matrix. However, when only local network information is available, 
only sufficient conditions for connectivity can be verified. One such condition is the requirement 
that \J\fj (t) n J\fi^r > 1, which can be locally verified by agent i with knowledge of only Qi^r- 
Since this condition is only sufficient but not necessary for connectivity preservation, we need 
a mechanism to check connectivity for those edges in the set 

C{t) = {it,j)e£it) : |A/',(t)nA/;„,(t)| = l}. 

of critically connected edges, for which the sufficient condition does not hold. 

The proposed mechanism relies on a the concept of a maximum consensus. In particular, 
consider a graph Q{t) = {V,£{t)) at time t > and for any G C{t) associate a scalar 

variable x^^'^\s) G M with every node k e V. Assume that the variables x^^'^\s) are randomly 

'"since the indices of all nodes in the network are distinct natural numbers, this definition results in a unique assignment. 
"a pair of nodes [i, k) is a nonedge of Q if (i, k) ^ £ (Q). 
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initialized and run the following maximum consensus update 

x^^'^^ {s + l)= max {xp^ (s)} (15) 

on the graph Q{t) — obtained by virtually disabling the link via blocking commu- 
nication through it. Then, the network Q(t) — is almost surely connected if and only if 
the variables x^i^'''\s) for all A; G V converge to the common value maxk x^f|^'^'^ (0). Note that 
convergence in this case takes place in finite time that is upper bounded by the diameter of the 
network [|T8l . This idea can be extended to simultaneous verification of multiple link deletions 
in C{t). In fact, since every edge is assigned a unique master agent, we can partition the set 
C(t) in to |V| disjoint subsets C{t) fl I'i(t) for all i E V. This allows us to define the sets 
'Pki = {^k'''^ i^) '■ (^'i) ^ ^(^) containing all variables of agent k that have as a master 

agent i. A simple schematic of the proposed construction is shown in the following table: 





cnVi 


cnV2 


1 

2 


Pii = {x^^')} 

V21 = {xS''^} 


Pi2 = {xf"^} ... 

V22 = {xf''^} ... 



Note that the second subscript i in the set Vki denotes that master agent for the variables 
contained in Vki- Therefore, agent k initializes only those variables in the set Vkk- Finally, 
stack all variables in the set Vki in a vector Xfcj(s) G ]RI'^(*)^^»(*)I and denote by [xki{s)]^^ j^ the 
scalar state associated with edge E C{t) r\Vi(t). Using the notation defined above, we can 
simultaneously verify connectivity for all edges in C{t) by a high-dimensional consensus. For 
this, every agent k initializes randomly all vectors Xfcj(O) E mI'^(*)^^»(*)I for all masters i E V 
and updates the vectors Xfcj(s) E ]RI'^(*)'~'^*(*)I as follows: 

Case I: If k is not a neighbor of the master agent i, i.e., if A; ^ Mi, then it updates the vectors 

Xfci(s) as 

Xfcj(s + 1):= max {xfcj(s), x/i(s)} , (16) 
where the maximum is applied elementwise on the vectors. 

Case II: If A; is a neighbor of the master agent i, i.e., if A; G Mi, then it virtually removes link 
{k,i) and updates the entry ['x.ki{s)](k,i) as 

[xfci(s + l)](A;,i) := max {[:s.ki{s)](^k,i),[^ii{s)](^k,i)} , (17) 
leMkit)\{i} 
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Algorithm 2 Connectivity verification 



Require: Xij-(O) G M}mnv,{t)\ f^j. j g V ; 
1: for s = 1 : r do 
2: Update Xij(s + 1) by ([T6|)-([T9|); 
3: end for 



4: Compute S,{t) by (20); 



while for all other links (j, i) G C{t) fl with j ^ k it updates the entries [Kki{s)]{j,i) as 

Ms + := max {[xfci(s)](j-i), [x/i(s)](j-i)} . (18) 

Case III: For the variables Xfcfc(s) for which is the master, it virtually removes the links 
{k,j) G C(t) r\Vk{t) and updates the entries [xfcfc(s)](fcj) as 

[xfcfc(s + l)](fc,i) := , max { [xfcfc(s)](fcj), [xikis)](^kj)} ■ (19) 



The high-dimensional consensus defined by (16)-(19) converges in a finite time r > [18] 



When this happens, node k requests the entries [xifc(r)](fc j) from all its neighbors i G A/fc(t) for 
which {k,i) G C{t)r]'Dk{t) and compares them with [xfcfc(r)](fc j). Since, violation of connectivity 
due to deletion of {k,j) would result in nodes k and i being in different connected components, 
if [xfcfc(s)](fc j) = [xjfc(s)](jt j) then the network Q (t) — {k,i) would still remain connected. Hence, 
we can define the set 

Sk{t) ^ {{k,t) G C{t)nVk{t) : [Xfcfc(r)](fc,i) = [^ik{r)]^k,i)} , (20) 

containing the edges in C{t) fl Vk{t) whose removal does not disconnect the network. 

B. Most Beneficial Local Action 

To solve Problem [T] via the iterative algorithm proposed in (|4]), we need to add or delete 
an edge (z, j) that minimizes the spectral pseudometric dK{S{Q±[ij){t)), S*) at every time step 
t. For this, let SD,{t) = cIk {S {g{t)) , S*) denote a local copy of the spectral distance of the 
graph Q(t) that is available to agent i, so that initially SDi{0) = SD(0) for all agents i E V. 
The quantity SD(0) can be computed in a distributed way by means of distributed averaging. 



according to Theorem 3.2 Then, the key idea is that every master agent i computes the spectral 
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distance SD±(^ij){s) = cLk ("S" ,5**) resulting from adding a link G Ai{t) or 



deleting a link {i.j) G Siif). Computation of this distance relies on Theorem 3.4 and requires 
that agent i has knowledge of the structure of its neighborhoods Qi.r only, for r = yK/2\. For 
all possible local edge additions or deletions, master agent i determines the most beneficial one 

(z, ^ argmin {SD±^,,,){t) - SD^it)] . 

(*,i)eA(t)u5i(t) 

Note that the minimization above may result in multiple edges having the same optimal value. 
Such ties can be broken via, e.g., a coin toss. Then, the largest decrease in the error associated 
with the most beneficial edge {i,j*{t)) becomes: 




SD±(ij^^{t), if min(ij)e^^(t)u5,(t){5L'±(i,i)(t) - SD,{t)} < 
otherwise 



for a large constant D > 0. In other words, SDi{t) is nontrivially defined only if the exists a 
Unk adjacent to node i that if added or deleted decreases the error function SD{t). Otherwise, a 
large value D > is assigned to SDi{t) to indicate that this action is not beneficial to the final 
objective. Finally, for each node i, we initialize the state vector 

b,(0)^[zjT(t) SD,{t) m(z,j;(t))f , 

containing the best local action (i,j*{t)), the associated spectral pseudodistance SDi(t), and the 
vector of resulting moments 

In the following section, we discuss how to compare all local actions hi(t) for all nodes i eV 
to find the best global action that minimizes the spectral pseudometric. 

C. From Local Information to Global Action 

In order to obtain the overall most beneficial action, all local actions need to be propagated 
in the network and compared against each other. For this, every agent i communicates with its 
neighbors and updates its desired action bj(s) with the action bj(s) corresponding to the node 
j that contains the smallest distance to the target moments [bj(s)]3 = SDi{t), i.e., 

bj(s + l) = bj(s), where 

j = argminfcg_^^(i){[bi(s)]3, [bfc(s)]3}. 
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Algorithm 3 Globally most beneficial action 



Require: b,(0) ^ j*(t) SD,{t) m(z, j7(t))]^; 
1: for s = 1 : r do 

2: hi{s + 1) := hj{s), with j = max{argminfcg_v-,(t){[bi(s)]3, [bfc(s)]3}; 
3: end for 

4: if [bi(r)]3 < D then 



5: Update M(t + 1), mi(t + 1) and SDi{t + 1) according to (|2T)-(|24]); 

6: else if [bi(r)]3 = D then 

7: No beneficial action. Algorithm has converged; 

8: end if 



In case of ties in the distances to the targets [bj(s)]3, then the node with the largest index is 
selected (line 2, Alg. [3]). Note that line 2 of Alg. |3]is essentially a minimum consensus update 
on the entries [bj(s)]3 and will converge to a common outcome for all nodes in finite time 
r > 0, when they have all been compared to each other. When the consensus has converged, if 
there exists a node whose desired action decreases the distance to the target moments, i.e., if 
[bj(s)]3 < D (line 4, Alg. |3]), then Alg. [3] terminates with a greedy action and node i updates 
its set of neighbors A/'i(t + 1) and vector of moments uii{t + 1) (line 5, Alg. |3|. If the optimal 
action is a link addition, i.e., if [bj(r)]2 ^ N'i{t), then 

+ :=M(t)U{[b,(r)]2}. (21) 

On the other hand, if the optimal action is a link deletion, i.e., if [bi(r)]2 G Ni{t), then 

Mi{t + l) :=M(t)\{[b,(r)]2}. (22) 

In all cases, the moments and error function are updated by 

m,(t + 1) := [[b,(r)]4 . . . [b,(r)]4+K]^ (23) 

and 

m(t + l):=[b.(r)]3, (24) 

respectively. Finally, if all local desired actions increase the distance to the target moments, i.e., 
if [bi(r)]3 = D (line 6, Alg. |3|, then no action is taken and the algorithm terminates with a 
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network topology with almost the desired spectral properties. This is because no action exists 
that can further decrease the distance to the target moments. 

D. Synchronization 

Communication time delays, packet losses, and the asymmetric network structure, may result 
in runs of the algorithm starting asynchronously, outdated information being used for future 
decisions, and consequently, nodes reaching different decisions for the same run. In the absence 
of a common global clock, the desired synchronization is ideally event triggered, where by a 
triggering event we understand the time instant that messages are transmitted and received by 
the nodes. For an implementation of such a scheme see ||T9ll . 

V. Numerical Simulations 

In the following numerical examples, we illustrate the performance and limitations of our iter- 
ative graph process. The objective of our simulations is to find a graph whose Laplacian spectral 
moments match those of a desired spectrum. In each example, we analyze the performance of 
our algorithm and study the spectral and structural properties of the resulting graph. 

Example 5.1 (Star vs. Two-Star Networks): In our first two simulations, we try to find graphs 
that match the spectral moments of (i) a star graph and (ii) a two-star graph (Fig. [3]). The Lapla- 
cian spectral moments of a star network with 10 nodes are: {mk)\^i = (1.8, 10.8, 100.8, 1000.8, 10000.8). 
Starting with a random graph on 10 nodes, we run our distributed algorithm to iteratively add 
and delete edges that minimize the spectral pseudodistance. We observe, in Fig. [2[ that the 
spectral pseudodistance evolves towards zero in 45 steps. We also verify that, although we are 
only controlling the first five spectral moments of the Laplacian matrix, the resulting network 
structure is exactly the desired star topology. This indicates that a star graph is an extreme case 
in which the graph topology is uniquely defined by their first five Laplacian spectral moments. 

In our second simulation, we consider the two-star network with 20 nodes in Fig. |3] (a). 
The Laplacian spectral moments of this graph are {'mk)l^i = (1.9, 12.8, 133.6, 1480, 16590). We 
observe in Fig. [2] how, after running our iterative algorithm for 94 iterations, our graph process 
stops in a graph topology with a spectral pseudodistance very close to zero (in particular, 5.2e— 2). 
The resulting topology, represented in Fig. [3] (b), is very close to the desired two-star network. 
This topology is a local minima of our evolution process because we could transform it into 
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Iteration step, t 

Fig. 2. Convergence of the spectral pseudodistance dk{St, S") for the star graph (blue plot) and the two-stars graph (red plot), 
where S* is the spectrum of the desired graph and St is the spectrum of Qt. 

our optimal two- star graph by two simple operations: (1) Adding an edge connecting nodes u 
and V (Fig. |3] (b)), and (2) removing edge {u,w). On the other hand, one can verify that step 
(1) would increase the spectral pseudodistance; therefore, our greedy evolution process does not 
follow this two-steps path. Despite this limitation, our final topology is remarkably close to the 
two-star network and their eigenvalue spectra are very similar, as shown in Fig. |4j 

Example 5.2 (Chain vs. ring networks): In the next two simulations, we try to find graphs 
that match the spectral moments of (i) a ring graph and (ii) a chain graph. Starting from a 
random graph, we run our iterative algorithm to match the spectral moments of a chain graph 
with 20 nodes, {nik)\^i = (1.9,5.6, 18.4,63.6,226.4). In this case, the spectral pseudodistance 
converges to zero in finite time and the final topology is exactly the desired chain graph. On 
the other hand, if we try to match the spectral moments of the ring graph in Fig. |5] (a), with 
{mk)l.^i = (2,6,20,70,252), an exact reconstruction is very difficult to achieve. In Fig. [s] (b), 
we depict the graph returned by our algorithm, after 83 iterations. Note that since we are only 
allowing local structural modifications in our graph process, it is hard for our algorithm to 
replicate long cycles in the graph. On the other hand, although the structure of the resulting 
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Fig. 4. Empirical cumulative distribution functions for the eigenvalues of the two-stars graph (blue) and the graph returned by 
our algorithm (red). The subgraph in the lower right corner shows the CDF's around the origin. 
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Fig. 5. In (a) we observe a ring graph with 20 nodes. The topology returned by our iterative algorithm can be observed in (b). 
1 
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Fig. 6. Empirical cumulative distribution of eigenvalues for the ring graph with 20 nodes (blue plot) and the topology returned 
by our algorithm in Fig. |5](b) (red plot). 



network is not the desired ring graph, its eigenvalue spectrum is remarkably close to that of a 
ring, as we can see in Fig. |6] 
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The above examples illustrate two limitations of our algorithm, namely, the existence of local 
minima in the graph evolution process and the inability of our algorithm to recover long cycles. 
Despite these limitations, our algorithm is able to find graph topologies with eigenvalue spectra 
remarkably close to the desired ones by matching five spectral moments only. Furthermore, the 
resulting topologies are structurally very similar to the desired ones, indicating that the spectral 
moments of the Laplacian matrix contains rich information about the structure of a network. In 
the next two examples, we show how our algorithm is also able to efficiently generate graphs 
matching the spectral properties of two popular synthetic network models: the Small-World EOl 
and the Scale-Free J2II networks. 

Example 5.3 (Small-Worlds): The small-world model was proposed by Watts and Strogatz 
[|20ll to generate networks with high clusterin^^ coefficients and small average distance. We can 
generate a small-world network by following these steps: (1) take a ring graph with n nodes, (2) 
connect each node in the ring to all its neighborhoods within a distance k, and (3) add random 
edges with a probability p. In this example, we generate a small-world network with n = 40, 
k = 2, and p = 3/n. The first three spectral moments of a random realization of this network are 
("^A:)fc=i = (6.55,51.9,457). Then, we run our algorithm to generate a graph whose first three 
spectral moments are close to those of the small-world network. After running our algorithm for 
78 iterations, we obtain a graph topology with a spectral pseudodistance very close to zero (in 
particular, 1.7e — 3) and an eigenvalue spectrum remarkably similar to that of the small- world 
network, as shown in Fig. |7} 

Example 5.4 (Power-Law): Another popular model in the 'Network Science' literature is the 
scale-free network. This model was proposed by Barabasi and Albert in [I2T1 to explain the 
presence of heavy-tailed degree distributions in many real-world networks. In this example, 
we generate a random power-law network with n = 50 nodes and m = 4, where m is a 
parameter that characterizes the average degree of the resulting network (see [l2T1l for more 
details about this model). A random realization of this network presents the following sequence 
of moments: {mk)\^i = (7.72, 111, 2.81e3, 9.70e4, 3.82e6). Then, after running our algorithm 
for 98 iterations, we obtain a graph topology with a spectral pseudodistance very close to zero 
(in particular, 5.5e — 2). The eigenvalue spectrum of the resulting topology is remarkably similar 

'^The clustering coefficient of a network is a measure of tiie number of triangles present in tiie network. 
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to that of the small- world network, as shown in Fig. [8j Furthermore, we can compare the degree 
sequences of the power-law network and the topology generated by our algorithm. We compare 
these sequences, sorted in descending order, in Figj9} We observe how the degree sequence of the 
topology obtained in our algorithm is remarkably close to that of the power-law network. This 
indicates that the spectral properties of a network contains rich information about the network 
structure, in particular, the first five spectral moments seems to highly constrain many relevant 
structural properties of the graph, such as the degree distribution. 

VI. Conclusions and Future Research 

In this paper, we have described a fully decentralized algorithm that iteratively modifies 
the structure of a network of agents with the objective of controlling the spectral moments 
of the Laplacian matrix of the network. Although we assume that each agent has access to 
local information regarding the graph structure, we show that the group is able to collectively 
aggregate their local information to take a global optimal decision. This decision corresponds to 
the most beneficial link addition/deletion in order to minimize a distance function that involves 
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Fig. 8. Empirical cumulative distribution for the eigenvalue spectrum of the power-law graph in Example |5.4| (blue) and the 
topology resulting from our algorithm (red). 
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Fig. 9. Degree sequences (in descending order) of the power-law graph in Example |5.4| (blue) and the topology resulting from 
our algorithm (red). 
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the Laplacian spectral moments of the network. The aggregation of the local information is 
achieved via gossip algorithms, which are also used to ensure network connectivity throughout 
the evolution of the network. 

Future work involves identifying sets of spectral moments that are reachable by our control 
algorithm. (We say that a sequence of spectral moments is reachable if there exists a graph whose 
moments match the sequence of moments.) Furthermore, we observed that fitting a set of low- 
order moments does not guarantee a good fit of the complete distribution of eigenvalues. In fact, 
there are important spectral parameters, such as the algebraic connectivity, that are not captured 
by a small set of spectral moments. Nevertheless, we observed in numerical simulations that 
fitting the first four moments of the eigenvalue spectrum often achieves a good reconstruction of 
the complete spectrum. Hence, a natural question is to describe the set of graphs most of whose 
spectral information is contained in a relatively small set of low-order moments. 



Appendix 

Theorem A. 1: Consider two undirected (possibly weighted) graphs Gi and G2 with (real) 
eigenvalue spectra S {d) = {A^^^ < ... < A^^^^} and 82(01) = {Xf^ < ... < A^f^}. Then, 
Af ^ = Af ^ for all 1 < i < n if and only if (Gi) = (G2) for < A; < n - 1. 

Proof: The theorem states that the spectrum S (A) = {Aj}"^^ of any n x n symmetric 
matrix A is uniquely characterized by its first n — 1 spectral moments. First, we use Cayley- 
Hamilton theorem to prove that the first n — 1 spectral moments of the spectrum S characterize 
the whole infinite sequence of moments {nik {S))'^^q, as follows. Let (A) = det (A/„ — A) — 
A" + ctn-iA""^ + ... + CKo, be the characteristic equation of A. Then, from Cayley-Hamilton, we 
have (^4) = 0. Multiplying (p (A) by ^A^, and applying the trace operator, we have that, 

-Trace \A'(f> (A)] = -Trace + q;„_i -Trace + ... + ctQ-Trace (A') 

n ^ ^ n ^ ' n ^ ' n ^ ' 

= mt+n {A) + an-imt+n-i (A) + ... + aomt {A) = 0, 
for all t eN. Therefore, given the sequence of moments (m^ (>l))^Io, we can use the recursion 

mt+n (A) = -an-imt+n-1 (A) - ... - aiuit+i {A) - aorrit (A) , 
to uniquely characterize the infinite sequence of moments {ruk (^))^o- 
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Second, we prove that the infinite sequence of moments (m^ (A))^q uniquely characterizes 
the eigenvalue spectrum. Let us define the spectral measure of the matrix A with real eigenvalues 
Ai < A2 < ... < A„, as 

n 

1=1 

where 6 (•) is the Dirac delta function. In what follows, we prove that the spectral measure of A is 
uniquely characterized by its infinite sequence of spectral moments using Carleman's condition 
[|22|. Since there is a trivial bijection between the eigenvalue spectrum of A and its spectral 
measure, uniqueness of the spectral measure imply uniqueness of the eigenvalue spectrum. 
Carleman's condition states that a measure /i on M is uniquely characterized by its infinite 

sequence of moments {Mk (z^))^! if (i) Mk (/i) < 00 for all k eN, and (ii) 

00 

s=l 

In our case, the moments of the spectral measure yU^ are 

r+oo 

Mk (iJ'a) = / x''diJ,A (x) 



^ A,^ = n rrik {A) 



i=l 



These moments satisfy: (z) Mk (ha) < '^A^ < 00, for any finite matrix A, and (//) 



s=l s=l \ i=l 

00 



> 



s=l 

00 



00, 
s=l 



for any A 7^ 0. As a consequence, the spectral measure of any finite matrix A ^ with real 
eigenvalues is uniquely characterized by {Mk (/^A))fclo- Since, Mk (/U^) = n rrik {A), we have 
that the sequence of moments (m^ (A))^Iq uniquely characterizes (M^. (/i^))^Q. Therefore, the 
sequence of moments {mk {A))^Z^ uniquely characterize the spectral measure and the real 
eigenvalue spectrum S = {Aj}"^^. ■ 
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